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Abstract 

A comparison is made between the off-diagonal and helicity spin bases for 
top quark pair production at the FNAL Tevatron. In the off-diagonal basis, 
92% of the top quark pairs are in the spin configuration up-down plus down- 
up, whereas in the helicity basis only 70% are left-right plus right-left. The 
off-diagonal basis maximizes the spin asymmetry and hence the measured 
angular correlations between the decay products, which are more than twice 
as big in this basis as compared to the helicity basis. In addition, for the 
process qq —» tt, we give a very simple analytic expression for the matrix 
element squared which includes all spin correlations between the production 
and subsequent decay of the top quarks. 
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Since the discovery [|l| of the top quark in 1995, a number of authors [Q-Q] have revis¬ 
ited the question of spin correlations in top quark pair production at the FNAL Tevatron. 
(References to the extensive literature of earlier works can be found in these recent papers.) 
With its high mass of about 175 GeV, the top quark decays before it hadronizes [|J. Thus, 
the decay products of a top quark produced in a definite spin state will have characteristic 
angular correlations. The reason for these new studies is the realization that the number 
of like-spin and unlike-spin top quark pairs can be made significantly different by an ap¬ 
propriate choice of spin basis. Both Mahlon and Parke [2|] and Stelzer and Willenbrock § 
discussed the spin asymmetry using the helicity basis for the Tevatron. However, the top 
quarks produced at the Tevatron are not ultra-relativistic; therefore, the helicity basis is 
probably not the optimal basis for these studies at this machine. Consequently, Ref. j2| 
went on to consider the beamline basis, which is more suitable for spin studies near thresh¬ 
old, and found larger effects at the Tevatron in this basis. The question remained, however: 
what is the optimal spin basis for these correlation studies? We now have an answer to this 
question. 

Recently Parke and Shadmi [[| studied the spin correlations for the process e + e“ —> tt 
and showed that you can choose a spin basis in which the like spin components, up-up 
(UU) and down-down (DD), identically vanish. They have called this spin basis the off- 
diagonal basis. In a footnote these authors briefly applied this result to qq —> tt, which is 
the dominant top quark production process at the Tevatron. The purpose of this letter is 
to study the qq —> tt process in more detail, comparing the off-diagonal basis to the more 
traditional helicity basis. Although the type of analysis described in Ref. 0 can also be 
applied to the gg initial state, the dominant tt production mechanism at the LHC, we find 
that no significant improvements over the helicity basis are possible at the LHC. At the 
Tevatron^] however, the process qq —> tt dominates, accounting for approximately 88% of 

throughout this paper, we take the center of mass energy for the pp collisions to be 2.0 TeV. We 
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the total tt cross section. Thus, we will concentrate our analytic discussions on that process. 
Our numerical results, however, include both the qq and gg initial states. 

In this letter, we present the production density matrix for qq —> tt in the off-diagonal ba¬ 
sis and contrast it with the known result for the helicity basis. Then we obtain a surprisingly 
simple and compact expression for the production and decay of a pair of top quarks from 
a quark-antiquark initial state which includes all of the correlations among the particles. 
We also show that the interference terms for the off-diagonal basis are substantially smaller 
than for the helicity basis at the Tevatron. Finally, we apply this basis to spin correlation 
studies at the Tevatron, where we find that in the off-diagonal basis, 92% of the tt pairs 
produced at the Tevatron have unlike spins. This represents a significant improvement over 
the helicity basis, in which only 70% of the tt pairs have unlike hclicities. As a result, the 
correlations obtained in the off-diagonal basis are more than twice as big as those in the 
helicity basis. 

In Fig. [I] the spatial part of the top quark momenta (t,t) and spin vectors (s,s) are 
given for the process qq —> tt in the zero momentum frame (ZMF) of the incoming quarks. 
What was shown in Ref. |J is that the spin projections of the top-antitop pair in qq —> tt 
are purely up-down and down-up if the spin vectors make an angle ip with respect to the 
beam axis. This angle is given by 


tan-0 = 


/3 SqtCqt 


( 1 ) 


i-/? 2 4’ 

where f3 is the speed of the top quarks in the ZMF.0 Throughout this paper we use the 
notation Sij and cy,- to denote the sine and cosine of the angle between the momenta of 
particles i and j in the ZMF: hence, c qt is the cosine of the top quark scattering angle (often 
denoted by cos#*). Note that near threshold the spin vectors are aligned along the beam 


use a top quark mass of 175 GeV, W boson mass of 80 GeV, and employ the MRS(Rl) structure 
functions fij evaluated at the scale Q 2 = rriyy. 

2 The angle ip is related to the angle a introduced by Tsai (8| for e + e _ —* t + t~ . 
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psl 2-^3% 0 


( 2 ) 


direction and that at very high energies they are aligned along the direction of the top and 
antitop momenta. 

For the off-diagonal basis, the production density matrix averaged over the color and 
spin of the incoming quarks is given byQ 

0 0 0 0 

0 2 -/ 3 2 4 phi 0 

0 

0 0 0 0 

The matrix element for the production of a t quark with spin A and a t quark with spin 
A is -M a x, and g s is the strong coupling constant. The ordering of the columns and rows 
in (Q) is (UU,UD,DU,DD). This production density matrix is very simple because in the 
off-diagonal basis the amplitudes with like spins, UU and DD, vanish identically. Also note 
that the non-diagonal terms have an explicit factor of f3 2 . 

In contrast, the same production density matrix in terms of the helicity basis reads || 


Y,m xx m\,- x ,= 



4/7 2 

SqtCqt /7 

SqtCqt /7 

s %h 2 

at 

SqtCqt /7 

2 - S 2 

z b qt 

s 2 

S qt 

SqtCqt /7 

9 

SqtCqt/ 7 

s 2 
b qt 

9 — s 2 

Z b qt 

SqtCqt/ 7 


4/7 2 

b qtCqt / 7 

SqtCqt/l 

4/7 2 


(3) 


where 7 = (1 —j3 2 ) 1 / 2 is the usual Lorentz boost factor. The columns and rows of this matrix 
have been ordered (RR,RL,LR,LL), with obvious notations for right and left hclicities. 


3 The phases of the non-diagonal terms of the production density matrix are dependent on many 
of our conventions, which we have chosen to make the phases as simple as possible. We take 
the 31 plane to be the scattering plane with the top quark direction the 1 axis. We use the 
conventions of [^] with the spinor products singular along the minus 2 direction and (k + \q —) = 
{[q 3 (k 0 +k 2 ) - k 3 (q 0 +q 2 )] + i[h(qo+q 2 ) ~ qi(ko+k 2 )]j[(k 0 +k 2 )(qo+q 2 )} / for positive energy 
light-like vectors k and q. 
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In the off-diagonal spin basis, the non-diagonal terms of the production density matrix 
are at least a factor of f3 2 times smaller than for the hclicity basis. For the Tevatron at 
2 TeV this corresponds to a factor of typically 0.3 to 0.4. Furthermore, the terms lying 
on the edges of the production density matrix in the hclicity basis are not very strongly 
suppressed compared to the diagonal terms, as 7 is only 1.2 to 1.3 at these values of f3 2 . 
These properties of the two matrices translate into smaller interference terms in the off- 
diagonal basis once the decays are included. 

Using either of the above production density matrices and the corresponding decay den¬ 
sity matrices, the total matrix element squared for the production and decay process^ 
qq —> tt —» W + b W~b —> evb gvb averaged over the initial quark’s color and spin and 
summed over the final colors and spins is given by 

El^l 2 = 

9s \ (r) q1 2 c eq c tiq) ~ /5( c /itT c ei) T P c qt ( c egT c //q) 4" c e/j) 

9" I ' 1 P Sqt) 7 2 (1 - p Cgt )(l - Pc,*) 

The factor T comes from the decay of the top quark (t —> W + b —> ev e b)\ 

T 9w / 2 _ 9 - , s ™ 2 t {l-& sh ) + {2e-v){l + c & ) 2 
4m?r ?^ mt } (2e ■ v - m* w y + (m w T w y ’ 

where c S b is the cosine of angle between e and b in the W + {= e + u) rest frame, 2e • v is 
the invariant mass of the positron and neutrino, (m*, T t ) and (mw, Tm) are H ie masses and 
widths of the top quark and lU-boson respectively, and gw is the weak coupling constant. 
Apart from the factor ( mtT t )~ 2 , which comes from the top quark propagator, T is just the 
matrix element squared for unpolarized top quark decay. Likewise, T is from the antitop 
decay (t —> W~b —> fiVfjb)'. 

4 We have assumed that the decays of both IF-bosons are leptonic. To change one or both of the 
VF-boson decays to a hadronic decay, replace the charged lepton with a down type quark and the 
neutrino with an up type quark. This will preserve all correlations. 
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m. gtv ( 2 9 mKi-gy+ ( 2 /x-p)(i + c /i5 ) 2 

T = tofr? (m > - 2 "' ^ 


( 6 ) 


( 2 ^ ■ v — mf v ) 2 + (mwIV ) 2 
where c.^, is the cosine of angle between // and b in the IE _ (= // + u) rest frame, and 2 /j • P is 
the invariant mass of the muon and anti-neutrino. Eq. (|J) agrees with the results of Kleiss 
and Stirling |10| for on mass shell top quarks and massless 6 -quarks[] independent of whether 
or not the IF-bosons are on or off mass shell. 

If we did not include the spin correlations between production and decay, the total matrix 
element squared would be simply g^TT{2 — ft 2 s 2 qt )/ 9. Thus, all of the correlations between 
the production and decay of the top quarks are contained in the second term inside the braces 
of Eq. (jdj). It is noteworthy that of the six final state particles, only the directions of the 
two charged leptons are required in addition to the directions of the t and t to fully specify 
these correlations. For the up-down spin configuration, the preferred emission directions for 
the charged leptons are (t + ms )/2 for the positron and (t + ms )/2 for the muon: for the 
down-up configuration they are (f — ms )/2 and (t — ms )/2 (see Fig. [I]). These light-like 
vectors make an angle u with respect to the beam axis given by 


sin a; = (3s qt , 


(7) 


and their energy components are r ym t (l ± (5c qt seco;)/ 2 . 

In the off-diagonal spin basis the interference terms (he. those terms in Eq. (|3j) coming 
from the non-diagonal pieces of the production density matrix) are 

atrf r . 


Io = 


9y 2 (l - f3cet){l - /3c Mt -) 2 L 


( c et CqtCeq P S qt){ C ^t c qt C fiq P S qt)/{^ ft S qt ) 


T ipet CqtC e q)(c^t CqtCeq) T S q ^(c e ^ T C eq C^q) 


( 8 ) 


whereas for the hclicity basis the interference terms are 


5 Inclusion of the finite mass effects for the 6-quarks would result in straightforward but messy 
modifications to Eqs. (j§) and (hi). The size of these effects is typically less than 1%. 
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(9) 


l h = 


gfTT 

97 2 (1 - Pcgt)(l - Pc^f) 


(J^Cqt C e q ) {PCqj ) 

Cet )(/3 C M t) + S qt( C eH T c etC^f) 


Here we see again that the interference terms are a factor of /3 2 smaller for the off-diagonal 
spin basis than the helicity basis. This point may be illustrated by plotting the distribution in 
X = X/ | j\4 1 2 : for each phase space point we compute the value of the interference term and 
divide by the total matrix element squared at that point. Because the total matrix element 
squared can range from 0 (maximal destructive interference) to 2X (maximal constructive 
interference), the variable X must lie in the interval (—oo,|]. The resulting differential 
distributions for both bases at the Tevatron is shown in Fig. [^. In the off-diagonal basis, 
da/dX resembles a sharp spike: in fact, 90% of the cross section comes from points where 
\X\ < 0.15. In contrast, only about half of the cross section comes from this region in the 
helicity basis. To enclose 90% of the cross section in the helicity basis, we must expand 
the range to \X\ < 0.35. Clearly, the interference terms are much less important in the off- 
diagonal basis than in the helicity basis. Thus, the off-diagonal basis provides a far superior 
description of the qq —> tt process at this accelerator. 

In Fig. [| we show the breakdown of the total tt cross section into like- and unlike-spin 
pairs using the off-diagonal basis versus the tt invariant mass for the Tevatron. We find 
that 92% of the tt pairs produced have unlike spins (UD+DU) in this basis. In contrast, 
only 70% of the tt pairs have unlike helicities.[j Note that we have chosen spin combinations 
which are insensitive to which beam donated the quark in qq —> tt. 

To observe the resulting correlations between the decay products of the top and the 
antitop, we proceed as described in Ref. p. Suppose that the ith decay product of the 
top quark is emitted at an angle 9 t with respect to the top spin axis in the top rest frame, 


6 The value of 67% unlike helicities we quoted in Ref. |2| is based on an older set of structure 
functions, which contained a somewhat larger gluon component. The distributions we consider are 
not significantly affected by the choice of structure functions. 
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and that the 7th decay product of the antitop is emitted at an angle 9j with respect to the 
antitop spin axis in the antitop rest frame. We tag the top quark in a particular event as 
having spin up if aycos^j > 0, and as having spin down otherwise. The angular distribution 
of the 7th decay product in this situation is 


da 


Or 


1 + |(1 — 2 P x ) ayay cos0* 


( 10 ) 


d(cosOi) 2 L 

where P x is the fractional purity of the unlike-spin component of the sample of ti events. 
The ay’s (ay’s) are the correlation coefficients from the decay distribution of a polarized top 
(antitop) [0 and take on the values a s = coj = 1, a v = a u = —0.31, and = —0.41 for 
the decay of a 175 GeV spin-up top quark. The ay’s for spin-down top quarks have opposite 
sign. The correlation coefficients for a spin up (spin down) antitop are the same as for a 
spin down (spin up) top quark. 

Because the W boson has primarily hadronic decays, it is useful to have a method to 
probabilistically determine which of the two jets in such a decay was initiated by the down- 
type quark. As explained in Ref. (2J, the jet which lies closest to the 5-quark direction 
as viewed in the W rest frame is most likely the d-type quark. The probability that this 
identification is correct is Py = |(2mf + 7m^)/^ + 2m^), and equals 0.61. The effective 
correlation coefficient for this “cT-type quark is given by cx^” = Pya?y + (1 — Py)ayj, or about 
-0.49. 


Since the coefficient of cos 6^ governs the size of the observable correlations, it is desirable 
to make it as large as possible. Thus, we should choose to work in the off-diagonal basis 
where 1 — 2 P x is —0.84 instead of the helicity basis where 1 — 2 P x is only —0.39. The other 
two factors, ay and ay depend on which top decay product is used to tag the top quark spin, 
and which antitop decay product angular distribution is generated. The largest correlations 
are clearly between the two charged leptons. For the same pair of decay products, the 
correlations are more than twice as large in the off-diagonal basis as in the helicity basis. 

In Fig. f| we show results of a first-pass Monte Carlo study of the correlations at the 
parton level without any hadronization or jet energy smearing effects included. We required 






all final state particles to satisfy the cuts px > 15 GeV, |?/| < 2. Plotted in this figure are the 
angular distributions for various antitop decay products for samples tagged as containing 
spin up or spin down top quarks. In the absence of correlations, the two curves in each 
section of the figure would he on top of each other. Even in the presence of the cuts, the 
off-diagonal basis still produces correlations more than twice as large as those in the hclicity 
basis. 

One advantage of the off-diagonal basis which is readily apparent from Fig. (| is that 
the cuts affect the spin-up and spin-down samples in a symmetric manner: i.e. the two 
curves remain reflections of each other about the point cos 6 = 0. Consequently, there is no 
systematic bias introduced between the two data sets when using this basis. The same is 
not true in the hclicity basis: our cuts are slightly more likely to exclude left-handed hclicity 
top quarks than right-handed, and the two angular distributions acquire different shapes [|| . 

In conclusion, we have shown that the off-diagonal basis of Parke and Shadmi enjoys 
several advantages over the more traditional hclicity basis when applied to a study of angular 
correlations in top quark pair production at the FNAL Tevatron. Firstly, the vanishing of 
the amplitude for the production of like spin top pairs from quark-antiquark annihilation 
leads to a total cross section consisting of 92% unlike spin pairs in the off-diagonal basis. 
This is significantly better than the 70% unlike hclicity pairs obtained in the hclicity basis. 
Secondly, most of the non-diagonal terms in the production density matrix vanish in the off- 
diagonal basis. Those non-diagonal terms which are not zero are suppressed by a factor of (3 2 . 
No such simplicity exists in this matrix in the hclicity basis. As a result, the contributions 
attributed to interference terms in the off-diagonal basis are far less important than those 
in the hclicity basis. Thirdly, the larger production asymmetry of the off-diagonal basis 
translates into angular correlations among the t and t decay products which are more than 
double those in the hclicity basis. And, finally, the kinds of experimental cuts which are 
typically imposed on collider data do not introduce a systematic bias between spin up and 
spin down top quarks in the off-diagonal basis. The same is not true in the hclicity basis. 
These advantages make the off-diagonal basis the basis of choice for tt correlation studies at 
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the Tevatron. 
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FIG. 1. The relevant angles and vectors in the zero momentum frame of the initial qq pair for 
the off-diagonal basis of Parke and Shadmi. The top quark is produced at an angle 9* with respect 
to the beam axis (cos 9* = c q t). The spin vector s makes an angle i/i (given by Eq. (jl])) with respect 
to the beam axis. The vectors (t ± ms)/ 2, where m is the top quark mass, indicate the preferred 
emission directions for the charged lepton or down-type quark from the decaying W + (see Eq. ©)• 
The vectors describing the antitop lie back-to-back with the corresponding top quark vectors. 
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FIG. 2. The relative importance of the interference terms in the off-diagonal and helicity 
bases in qq —> tt for the Tevatron at s/s = 2 TeV. Plotted is the differential distribution in 
T = Z/J]|.A/f| 2 , the value of the interference term (Eq. @ or (||)) normalized to the square of 
the total matrix element (Eq. (]3|)). In the off-diagonal basis, 90% of the cross section comes from 
phase space points where |Z| < 0.15, whereas in the helicity basis only 50% of the cross section 
comes from this region. 
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FIG. 3. Differential cross section for tt production as a function of the tt invariant mass M t j for 
the Tevatron with center-of-mass energy 2.0 TeV, decomposed into UD+DU and UU+DD spins of 
the tt pair using the off-diagonal basis for both qq and gg components. 
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FIG. 4. Angular correlations in Tevatron tt events at \fs = 2 TeV using the off-diagonal 
basis. The data in each plot are divided into spin-“up” (solid) and spin-“down” (dashed) top 
quark components, determined by using the charged lepton from the t decay in (a)-(c), and the 
6-quark in (d). Plotted are the angular distributions with respect to the t spin axis in the t rest 
frame for the following t decay products: (a) the charged lepton, (b) the “d”-type quark, (c) and 
(d) the 6-quark. 





